Skip to content

feat(pgpm): pgpm transform — re-dial an existing module/workspace; wire --partition into pgpm export - #1569

Merged
pyramation merged 1 commit into
mainfrom
feat/pgpm-transform
Jul 31, 2026
Merged

feat(pgpm): pgpm transform — re-dial an existing module/workspace; wire --partition into pgpm export#1569
pyramation merged 1 commit into
mainfrom
feat/pgpm-transform

Conversation

@pyramation

Copy link
Copy Markdown
Contributor

Summary

Part B of constructive-planning#1340 (also closes the "wire the partition dial into pgpm export" TODO on constructive-planning#1329): the dials pipeline gets a second front door — existing pgpm modules, not just live databases.

pgpm transform --granularity <atomic|object|consolidated> \
  [--partition <partition.json>] [--naming directory|flat] \
  [--cwd <module-or-workspace>] [--out <dir>] [--write] [--check] [--dry-run]

Flow: loadModuleSource(moduleDir) (pgpm.plan → deploy scripts flattened in plan order, PGPM headers/tx wrappers stripped) → restructureExportRows (spec-derived paths, graph-derived requires, generated revert/verify) → optional partitionExportRows → complete pgpm package(s) written per partition.

New shared seam in @pgpmjs/export (one code path for transform and export; loadModuleSource is the reusable "module → flattened classified change list" helper intended for future pgpm import (dumps) and pgpm diff):

  • module-source.tsloadModuleSource, stripTransactionWrapper
  • partition.tsparsePartitionConfig (validated JSON matching PartitionConfig), partitionExportRows (runs partitionUnits, regenerates revert/verify per emitted change, keeps <pkg>:<path> cross-package deps and package-level requires)
  • catalog-check.tssnapshotCatalog / diffCatalogSnapshots, an order-insensitive structural catalog snapshot (schemas, tables, columns incl. order, constraints, indexes, functions, triggers, policies, RLS, grants) used by --check

Behavior notes:

  • Output packages are named <module>-<granularity> (or the partition package names) and written as siblings of the source module by default, so they can coexist in the same workspace; --out overrides the base dir. Overwriting the source module or any existing output requires --write.
  • Workspace mode iterates every module; --dry-run prints packages/changes/deps without writing; PartitionCycleError and dynamic-SQL warnings surface as clean CLI errors/warnings.
  • --check deploys the original module and the transformed package(s) (in dependency order) into two scratch databases and asserts the catalogs are structurally equivalent. Column order IS compared; per-class object sets are compared by identity+definition, so change granularity/naming can differ freely.
  • pgpm export --granularity ... --partition <file> goes through the exact same partitionExportRows path, emitting N packages with cross-package requires merged into each .control via preparePackage (SQL and GraphQL export modes).

Tests:

  • Unit: module loading/flattening/tx-stripping, partition config validation, out-dir resolution, overwrite guards, package dependency ordering.
  • Live-Postgres e2e (pgpm/cli/__tests__/transform-e2e.test.ts): fixture with 2 schemas, tables + FK, function, trigger, policy, grant, index → transform to each granularity → deploy → catalog equivalence with the original → pgpm verifypgpm revert leaves the DB clean; --check; --dry-run; partition split into pkg-app + pkg-security (splitRiders: grant) deployed in dependency order with catalog equivalence; round-trip consolidated → atomic → consolidated yields the same change set.
  • Export partition e2e added to export-granularity.test.ts (two packages, <pkg>:<path> requires, dependency-order deploy round-trip).

Known pre-existing failures (verified identical on a clean main worktree in this environment): export-utils/export-meta/export-flow/cross-flow-parity (7 tests) and repo-wide ESLint 9 flat-config lint — untouched.

Link to Devin session: https://app.devin.ai/sessions/98edd75e27db4328aec77c581e211092
Requested by: @pyramation

@pyramation pyramation self-assigned this Jul 31, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant